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Abstract 

Background: The training to become a dentist can create psychological distress. The present study evaluates the structure 
of the 'Perceived Stress Questionnaire' (PSQ), its internal consistency model and interrelatedness with burnout, anxiety, 
depression and resilience among dental students. 

Methods:Tbe study employed a cross-sectional design. A sample of Spanish dental students (n = 314) completed the PSQ, 
the 'Goldberg Anxiety and Depression Scale' (GADS), 'Connor-Davidson Resilience Scale' (10-item CD-RISC) and 'Maslach 
Burnout Inventory-Student Survey' (MBI-SS). The structure was estimated using Parallel Analysis from polychoric 
correlations. Unweighted Least Squares was the method for factor extraction, using the Item Response Theory to evaluate 
the discriminative power of items. Internal consistency was assessed by squaring the correlation between the latent true 
variable and the observed variable. The relationships between the PSQ and the other constructs were analysed using 
Spearman's coefficient. 

Results: The results showed a PSQ structure through two sub-factors ('frustration' and 'tenseness') with regard to one 
general factor ('perceived stress'). Items that did not satisfy discriminative capacity were rejected. The model fit were 
acceptable (GFI = 0.98; RSMR = 0.06; AGFI = 0.98; NFI = 0.98; RFI = 0.98). All the factors showed adequate internal consistency 
as measured by the congeneric model (>0.91). High and significant associations were observed between perceived stress 
and burnout, anxiety, depression and resilience. 

Conclusions: The PSQ showed a hierarchical bi-factor structure among Spanish dental students. Using the questionnaire as 
a uni-dimensional scale may be useful in perceived stress level discrimination, while the sub-factors could help us to refine 
perceived stress analysis and improve therapeutic processes. 
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Introduction 

The training to become a health professional can create 
psychological distress and symptoms of burnout, which may have 
adverse consequences for one's personal and professional life [1— 
5] . Dentists experience high levels of work-related stress, which 
begins during professional training [6]. Furthermore, the preva- 
lence of burnout among dentistry students is high, with a 
significant relation found between the syndrome and a student's 
academic performance, use of medication and thoughts of 
dropping out of the course [6]. Dental students complain of 
exhaustion resulting from the high levels of anxiety generated by 
exams, limited free time available for relaxation and the stress 
associated with having to adapt to the requirements of clinical 



practice [7-9]. As a result, some new dentistry graduates exhibit 
alarmingly high levels of burnout [10]. 

'Stress' occurs when environmental demands overwhelm 
individuals' resources and threaten their personal well-being. It 
has been defined as the result of a relationship with the 
environment that individuals appraise as significant for their 
well-being and in which the demands exceed the available coping 
resources [11]. 'Burnout' is a response to the failure to cope 
adequately with chronic occupational stress and is an attempt to 
adapt to or to protect oneself from it [12]. This syndrome has 
classically been characterized by a state of exhaustion, cynicism 
and inefficacy [13]. 'Exhaustion' is the feeling of not being able to 
offer any more of oneself; 'cynicism' represents a distant attitude 
towards work, those served by it and other colleagues; and 
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'inefficacy' is the feeling of not performing tasks adequately or 
being incompetent. 

On the other hand, 'resilience' has been characterized as a 
dynamic and flexible process of adaptation to life changes that 
could serve as a protective factor against psychological distress and 
mental disorders [14,15]. It is the amount of personal strength, 
energy and motivation that enables an individual to cope with and 
recover from stress and to flourish when faced with adversity. Most 
of the proposed models to improve well-being in students and 
health professionals aim to enhance strategies for coping with 
stress using educational and environmental support as well as 
cognitive exercises that strengthen resilience skills [16,17]. 

From the early stages of their university studies, dental students 
show concern for the stress produced by their experiences with 
clinical practice. They develop intense and long-term interactions 
with clients and patients, which is characteristic of careers with 
high levels of psychological distress and burnout [18]. Experienc- 
ing burnout over a prolonged period is associated with adverse 
emotional consequences, such as anxiety and depression, and it 
can also negatively affect the quality of patient care [19-24]. As a 
result, being able to measure the stress process in dental students 
seems to be essential if we want to evaluate their well-being along 
with their period of professional training. 

The 'Perceived Stress Questionnaire' (PSQ) is a 30-item 
instrument that was developed by clinicians to quantify perceived 
stress [25]. It is one of the most used instruments that measure the 
stress process in psychosomatic research and has been associated 
with somatic complaints. In addition, it has demonstrated good 
predictive values in stress-related diseases [26-28]. The PSQ 
permits the subjective experience of perceived stressful situations 
and stress reactions to be assessed, emphasizing cognitive 
perceptions more than emotional states or specific life events. 
The general form of the instruction asks questions related to 'the 
last two years' and the recent form asks about situations taking 
place 'during the last month', potentially addressing chronic and 
acute relationships with stressful events and activities. It contains 
both positively and negatively formulated items in order to reduce 
acquiescent bias. Each item is answered using a four-point Likert- 
type scale, ranging from 1 ('almost never') to 4 ('almost always'). 
Higher scores indicate more severe perceived stress. Originally 
designed in English, this instrument has been translated to Italian, 
German and Spanish, and validated in populations of psychiatric 
inpatients and outpatients, nursing students, health workers, 
psychosomatic patients and health adults [25,27,28]. 

One key element of this questionnaire is that it includes different 
groups of stressful experiences, such as harassment, overload, 
irritability, lack of joy, fatigue, worries and tension, which were 
proposed in its original version. However, this structure was not 
replicated. The study by Sanz-Carrillo [27] revealed the dimen- 
sions of social acceptance, load, irritability, energy, fear and self- 
realization; while the Fliege's study [28] observed worries, tension, 
lack of joy and demands, differentiating the first three as internal 
stress reactions, while demands was classified as external stressors. 
So far, contemplating a general factor seems to be the most 
reasonable solution for its use. The original author proposed a 
linear algorithm ranging from 0 to 1 [25]. 

The previous inconsistencies may be due to differences among 
the samples that were used, which is in line with the idea that 
validating instruments in a specific population of interest is the 
most advisable option [29] . Nevertheless, the principal component 
analysis used as factor extraction method from r correlation 
matrices, along with the Kaiser role for the number of factors to 
retain, do not seem to be the best option to estimate factorial 



models using psychological variables as mentioned, and they are 
too often used out of force of habit [30]. 

Therefore, the goals of the present study were to evaluate the 
factor structure of the PSQ with dental students, together with its 
internal consistency model and its interrelatedness with the 
constructs of burnout, anxiety, depression and resilience. As a 
hypothesis, we expected high and significant correlations between 
all constructs. 

Methods 

Study Design 

We used a cross-sectional design by means of the application of 
a self-assessment survey. 

Setting, Sample and Ethics Statement 

The population consisted of Spanish dental students enrolled in 
Huesca (Nh=136) and Santiago de Compostela (Ns = 242), 
during the 2010-11 academic year. An 83.1% response rate 
(RR) to the surveys, which were sent to all prospective participants, 
resulted in a sample of n = 314 participants. The students did not 
receive any financial or credit compensation for participating in 
the study. No differences were found in RR based on sex 
(males = 81.4% vs. females = 83.8%; p = 0.576), campus 
(Huesca — 87.5% vs. Santiago de Compostela = 80.6%; 
p = 0.085) or age (participants Mn = 22.05; SD = 3.57 vs. non- 
participants Mn = 22.34; SD = 3.83; p = 0.551). 

A clinical psychologist trained two research assistants to 
administer the questionnaires as a battery in a paper-and-pencil 
format. The first page of the protocol identified the objectives of 
the study, participants, potential benefits and risks and the 
confidentiality of the data treatment. Each participant provided 
written informed consent before the commencement of the survey. 
The research assistants administered the survey in May 20 1 1 , two 
weeks before the period of final exams. After completion, the 
questionnaires were collected and kept in a sealed envelope to 
ensure the participants' anonymity. The Ethical Review Board of 
Aragon, Spain, approved the study protocol. This study followed 
Helsinki Convention norms and later modifications, the Declara- 
tion of Madrid of the World Psychiatric Association and the 
Uniform Requirements for manuscripts submitted to Bio-medical 
journals. 

Measures 

Socio-demographics. Data were collected on age, gender, 
stable relationship ('yes', 'no'), children ('yes', 'no'), campus 
('Huesca', 'Santiago'), distance from family home (km), residence 
('with parents', 'dormitory', 'shared flat', 'private flat'), scholarship 
('yes', 'no'), parental support perceived ('insufficient', 'good', 'very 
good'), weekly study hours, failed subjects over the previous period 
('none', 'one', 'two or more'), job ('yes', 'no') and year of study 
('first', 'second', 'third', 'fourth', 'fifth'). 

Perceived Stress Questionnaire (PSQ). The Spanish 
version of PSQ (described above) was used, specifically its "recent 
form" addressing the previous 30 days [25,27]. 

Goldberg Anxiety and Depression Scale (GADS). The 
participants completed the Spanish version of the GADS [31,32], 
which consists of a 9-item subscale that assesses symptoms of 
anxiety and a 9-item subscale that assesses symptoms of 
depression. The participants respond with either 'yes' or 'no', 
with one point scored for each positive response. A greater number 
of positive responses is associated with a greater likelihood of 
suffering from anxiety or depression. Each subscale provides a 
measure of the associated mental disorder. The convergence 
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validity of the GADS has demonstrated adequate values of 
sensitivity and specificity [32-34]. We used this short and friendly 
scale because it is recommended for the screening of anxiety and 
depression in large sample studies in the general population [31- 
32]. 

Connor-Davidson Resilience Scale (10-item CD- 
RISC). The participants also completed the Spanish version of 
the 10-item CD-RISC [35-36]. This scale is a self-report 
instrument that measures resilience on a 5-point Likert scale with 
responses that range from 0 ('not at all') to 4 ('almost always'). The 
final scores are obtained by summing the response to each of the 
items, with higher values indicating higher levels of resilience. The 
validity and internal consistency of the scale were adequate and 
positively related to variables such as sleep quality and mental 
health [35,36]. 

Maslach Burnout Inventory-Student Survey (MBI- 

SS). Subjects were given the MBI-SS in its validated Spanish 
version for students [13,37]. This questionnaire is the 'golden rule' 
for the evaluation of burnout, and this adaptation consists of 1 5 
items grouped into the three dimensions: five items corresponded 
to exhaustion, four to cynicism and six to efficacy. Responses are 
arranged in a 7-point Likert scale, scored from 0 ('never') to 6 
('always'). The scores of each dimension are obtained by adding up 
the responses to the corresponding items, with higher values 
indicating higher levels of each one. The questionnaire dimensions 
present an adequate structure and internal consistency [37]. 

Data Analysis 

SPSS vl9.0, FACTOR v9.02 and AMOS v7.0 statistical 
packages were used to conduct the analysis. 

Descriptives. Means, standard deviations, skewness, kurtosis 
and Mardia's coefficients [38] were calculated to evaluate the 
performance of the PSO_ items. 

Dimensionality. Polychoric correlation is advised for facto- 
rial analysis (FA) when the distributions of ordinal items are 
asymmetric, with excess of kurtosis or with high item-rest 
coefficients [39]. Thus, a polychoric correlation matrix was 
estimated with regard to the PSO_ items. We used parallel analysis 
(PA) [40] to identify the number of factors to include in the 
factorial solution, by replacing the raw data method [41] with 
optimal implementation based on minimum rank factor analysis 
[42], generating 500 random correlation matrices from permuta- 
tion of the raw data. With this analysis, a factor is considered 
significant if the associated eigen-value is greater than that 
corresponding to 95th percentile of the distribution of eigen- 
values derived from a random dataset. PA is considered the best 
available solution to decide the number-of-factors-to-retain 
[43,44]. We had previously verified the adequacy of the matrix, 
assessing the determinant, the KMO index and Barlett's test of 
sphericity [45]. 

Unweighted Least Squares (ULS) was the method used for 
factor extraction [46]. The ULS procedure does not provide 
inferential estimations for assessing model data fit based on the % 2 
distribution, but it has important advantages: it does not require 
any distributional assumptions; it is quite robust and usually 
converges because of its efficiency in terms of computation; and it 
tends to provide less biased estimates of the true parameter values 
[47]. Additionally, ULS is an appropriate choice for the case of not 
excessively large samples and shows good performance when 
working with polychoric matrices. In fact, ULS is consistent with 
the underlying variables approach from the Item Response 
Theory (IRT); it tends to supply accurate estimates even when 
models are large; and it provides better estimates than far more 
complex procedures [48-50]. The rotation method used was 



Promax, with a parameter of k = 4.00, given the correlated 
solution expected and using Raw Varimax as clever rotation start. 
We used Bender's simplicity index (S) and the loading simplicity 
index (LS) to evaluate factorial simplicity [51,52]. 

Although we used unrestricted item factor analysis at this stage, 
we actually propose a second order factor solution, as a perceived 
stress general factor (G), by hierarchical factor solution [53]. This 
specification, which is multidimensional but is able to reflect the 
essential uni-dimensionality of the data, prescribes a factor which 
reflects what is common to all of the items, working as a 
multidimensional semi-confirmatory factorial analysis [54]. The 
factor weights (w) and the % of variance explained in each item by 
means of communality values (h 2 ) were calculated. IRT param- 
eterization by multidimensional normal-ogive graded response 
model (derived from the assumption of normally distributed 
measurement error), showed us the pattern of item discriminations 
in each dimension [55]. The belonging factor was determined by 
means of the IRT discrimination (a n ), with those items with poor 
values being dismissed. Specifically, those items with a n values 
<0.65 were discarded, as well as those with a n values >0.65 in all 
of the latent dimensions at the same time [56]. 

We examined the fit of the proposed PSO_ model by CFA, 
applying ULS from a polychoric matrix, for the reasons stated 
above. From a general perspective, we used the goodness-of-fit 
index (GFI), the adjusted goodness-of-fit index (AGFI), the root 
mean square of the standardized residuals (RMSR), the normed- 
fit-index (NFI) and Bollen's relative-fit-index (RFI). GFI and AGFI 
refer to explained variance and values >0.90 are considered 
acceptable [57]. SRMR is the standardized difference between the 
observed and the predicted covariance, indicating a good fit for 
values <0.08 [58] . NFI measures the proportional reduction in the 
adjustment function when going from null to the proposed model 
and is considered acceptable when >0.90 [59]. RFI takes into 
account the discrepancy for the model evaluated and for the 
baseline model, and it is very good close to 1 [60] . All of them are 
perfectly valid for the ULS procedure. Taken together, they 
provide a reliable evaluation of the solution and additional 
information regarding absolute and incremental model fit. From 
an analytical perspective, standardized factor saturations and the 
explained variance were also considered. 

Reliability. We examined the internal consistency of the 
scales and sub-escales using congeneric, tau-equivalent and 
parallel models [61]. The congeneric model is the least restrictive 
and assumes that each individual item measures the same latent 
variable, with possibly different scales, degrees of precision and 
magnitude of error. The tau-equivalent model implies that 
individual items measure the same latent variable on the same 
scale and with the same degree of precision, but with possibly 
different degrees of error. The parallel model is the most restrictive 
measurement model; it assumes that all items must measure the 
same latent variable on the same scale, with the same degree of 
precision and with the same amount of error. We finally chose the 
model that fitted better with the data, applying the ULS method 
and establishing comparisons. The reliability value was estimated 
by squaring the implied correlation between the composite latent 
true variable and the composite observed variable, to arrive at the 
percentage of the total observed variance that was accounted for 
by the true variable [62] . Mean inter-item polychoric correlations, 
item-rest and mean item-rest correlations were also used, as well as 
the mean Spearman's R coefficients between the items over the 
belonging factor, calculated according to the Bayes 'Expected A 
Posteriori' (EAP) [63]. 

Convergence. Participant's scores on PSO_ factors calculated 
by EAP were also used in order to evaluate the degree of 
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association between them and regarding the other constructs by 
means of Spearman's R coefficients. 

Results 

In order to adhere to standards for data availability, the authors 
state that all materials used to produce the results in this paper will 
be made available upon request. This includes [64] : 1 .- The list of 
documents and data files that are needed in order for replication to 
be possible, 2.- A detailed list of what will be provided by the 
authors, and 3.- What steps, and in what sequence, the interested 
researchers need to take in order for this data to be made 
available. In addition, authors will post these materials on the 
group's website [65]. 

Participants 

Table 1 displays the general characteristics of participants. They 
comprised adults of European ethnicity between the ages of 18 and 
41 (Mn = 22.05; SD = 3.57), 70.70% of whom were women. 
Compared to students in Santiago de Compostela, students in 
Huesca lived further away from the family home (x — 72.53; 
df=2; p<0.001), were more likely to live in shared flats 
(% 2 = 14.79; df= 3; p = 0.002), were less likely to have received a 
scholarship (x 2 = 6.66; df= 1; p = 0.010) and failed a higher 
percentage of subjects over the previous exam period (% 2 = 7.33; 
df = 2; p = 0.026). Students in both Santiago and Huesca were 
similar with regard to the rest of the socio-demographic and 
occupational variables. 

Descriptives 

Descriptive statistics for all the PSQ_ items can be seen in 
Table 2. Some items presented skewness values >1.00, such as 
'you feel lonely or isolated' (item n° 5) and 'you find yourself in 
situations of conflict' (n° 6). Otherwise, some items showed kurtosis 
values < — 1.00, as 'you feel calm' (n° 10 reversed), 'you are under 
pressure from other people' (n° 19), 'you are afraid for the future' 
(n° 22) and 'you feel under pressure from deadlines' (item n° 30). 
Mardia's multivariate skewness and kurtosis coefficients were 
132.70 (p= 1.00) and 1,040.71 (p<0.001), respectively. 

Dimensionality 

The polychoric matrix of the PSQ_ items revealed that 78.6% of 
coefficients out of the diagonal were &0.30. The determinant was 
<0.001. KMO test had a value of 0.95 and Bartlett's statistic was 
4,780.50 (df=435; p<0.001). The PA identified a two-factor 
structure, explaining 54.9% of the variance [(ki = 12.23; 46.7% 
variance of the real data; 7.5% variance explained over the P 95 of 
the random samples); (X 2 = 2.39; 8.2% variance of the real data; 
6.9% variance explained over the P95 of the random samples)]. 
The hierarchical solution by two first order factors (Fj and F 2 ) and 
one second-order factor (G) exhibited good simplicity indices, such 
as S = 0.98 (P 99 ) and LS = 0.45 (P 99 ). ' 

Table 2 shows the weights over G, the rotated loading matrix 
over Fi and F 2 and the h 2 values. All the items loaded SO. 40 in G, 
except 'you feel you are doing things you really like' (n° 7 reversed; 
w g = 0.27). F! presented topics related to 'frustration', such as 'you 
are light-hearted' (n° 25 reversed; W| = 0.94), 'you feel discour- 
aged' (n° 20; W] =0.82) or 'you feel frustrated' (n° 12; Wj = 0.72). 
F 2 exhibited themes associated with 'tenseness', such as 'you have 
too many things to do' (n° 4; w 2 = 0.95), 'you feel that too many 
demands are being made on you' (n° 2; w 2 = 0.83), or 'you feel you 
are in a hurry' (n° 16; w 2 = 0.82). In general, h 2 values were high, 
with an average of 0.49. Table 2 shows the PSQ items in terms of 
IRT discrimination. Some items did not present sufficient values, 



Table 1. Characteristics of the participants (n = 314). 





Age, Md (SD) 


22.05 (3.75) 


Sex, females (%) 


222 (70.7) 


Stable relationship, no (%) 


158 (50.5) 


Children, none (%) 


300 (95.5) 


Distance from family home (%) 


<75 Km 


110 (35.0) 


75-150 Km 


103 (32.8) 


>150 Km 


101 (32.2) 


Place of residence (%) 


with parents 


38 (12.1) 


dormitory 


51 (16.2) 


shared flat 


183 (58.3) 


private flat 


42 (13.4) 


Scholarship, no (%) 


199 (63.4) 


Campus, Santiago (%) 


195 (62.1) 


Family support (%) 


insufficient 


20 (6.4) 


Good 


74 (23.6) 


very good 


220 (70.0) 


Weekly studying hours, Md (SD) 


37.27 (17.52) 


Failed subjects (%) 


None 


212 (67.9) 


One 


78 (24.6) 


two or more 


24 (7.5) 


Job, no (%) 


266 (84.7) 


Year of study (%) 


First 


62 (19.8) 


second 


63 (20.0) 


third 


60 (19.1) 


fourth 


69 (22.0) 


fifth 


60 (19.1) 



Md = Mean; SD = Standard Deviation; Number and percentage (%). 
doi:1 0.1 371 /joumal.pone.0087071 .t001 



such as 'you feel rested' (n° 1 reversed; ai = 0. 1 1; a 2 = 0.58), 'you 
are irritable or grouchy' (n° 3; &\ = 0.52; a 2 = 0.56), and also items 
n° 6 (a! = 0.50; a 2 = 0.08), n° 19 ( ai = 0.52; a 2 = 0.50) and n° 22 
(a.i — 0.43; a 2 = 0. 1 1), already mentioned. Additionally, 'your 
problems seem to be piling up' (n° 15; a! = 0.73; a 2 = 0.79) 
presented high values in both factors, so it was also dismissed. 

Figure 1 shows the PSQ_ hierarchical bi-factor structure using 
CFA from an analytical and standardized point of view. The two 
first order factors turned out to be highly influenced by G, with 
loadings over Fi = 0.96 and F 2 = 0.82. The item loadings with 
regard to their respective latent factor were high (Fi and F 2 ranges 
= 0.42 to 0.81 and = 0.38 to 0.85, respectively). In a general sense, 
the PSQ, hierarchical bi-factor structure presented adequate fit 
indices without using correlations between the error terms 
(GFI = 0.98; RSMR = 0.06; AGFI = 0.98; NFI = 0.98; RFI = 0.98). 

Reliability 

Table 3 shows the reliability models tested for the PSQ 
hierarchical bi-factor structure. The indices fitted best with the 
congeneric model in all of the latent factors. Based on the 
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Table 2. Descriptives, factorial solution, communalities and IRT parameterization of the PSQ items. 





Items 


Md 


SD 


skew 


kurt 


r i(t-i> 


G 


F, 


F 2 


h 2 


ai 


32 


1 . You feel rested (r) 


3.06 


0.90 


-0.69 


-0.33 


0.42 


0.52 


0.09 


0.48 


0.31 


0.11 


0.58 


2. You feel that too many demands are being made on you * 


2.65 


0.94 


-0.01 


-0.94 


0.52 


0.60 


-0.22 


0.83 


0.47 


-0.31 


1.14 


3. You are irritable or grouchy 


2.05 


0.84 


0.53 


-0.24 


0.58 


0.65 


0.37 


0.40 


0.50 


0.52 


0.56 


4. You have too many things to do * 


3.11 


0.86 


-0.57 


-0.59 


0.53 


0.65 


-0.30 


0.95 


0.58 


-0.46 


1.47 


5. You feel lonely or isolated * 


1.56 


0.81 


1.35 


1.05 


0.41 


0.44 


0.61 


-0.04 


0.35 


0.76 


-0.04 


6. You find yourself in situations of conflict 


1.66 


0.83 


1.12 


0.50 


0.43 


0.40 


0.44 


0.07 


0.24 


0.50 


0.08 


7. You feel you're doing things you really like (r) * 


1.72 


0.77 


0.81 


0.04 


0.22 


0.27 


0.67 


-0.26 


0.27 


0.78 


-0.30 


8. You feel tired * 


2.83 


0.90 


-0.17 


-0.93 


0.56 


0.69 


0.11 


0.65 


0.54 


0.17 


0.97 


9. You fear you may not manage to attain your goals * 


2.34 


0.95 


0.32 


-0.79 


0.51 


0.59 


0.54 


0.19 


0.47 


0.74 


0.26 


10. You feel calm (r) * 


2.58 


1.01 


-0.13 


-1.05 


0.52 


0.65 


0.52 


0.28 


0.55 


0.76 


0.41 


1 1. You have too many decisions to make * 


2.59 


0.80 


0.15 


-0.53 


0.42 


0.41 


-0.32 


0.71 


0.29 


-0.38 


0.84 


1 2. You feel frustrated * 


1.80 


0.83 


0.86 


0.19 


0.49 


0.58 


0.77 


-0.01 


0.58 


1.18 


-0.02 


13. You are full of energy (r) * 


2.63 


0.91 


-0.19 


-0.74 


0.41 


0.56 


0.68 


0.04 


0.51 


0.97 


0.06 


14. You feel tense * 


2.38 


0.91 


0.21 


-0.74 


0.67 


0.73 


0.34 


0.51 


0.62 


0.55 


0.82 


15. Your problems seem to be piling up 


2.18 


0.95 


0.40 


-0.75 


0.70 


0.75 


0.42 


0.46 


0.67 


0.73 


0.79 


16. You feel you're in a hurry * 


2.73 


0.94 


-0.23 


-0.85 


0.59 


0.69 


-0.09 


0.82 


0.57 


-0.14 


1.25 


17. You feel safe and protected (r) * 


2.29 


0.91 


<0.01 


-0.95 


0.40 


0.48 


0.70 


-0.06 


0.44 


0.94 


-0.08 


18. You have many worries * 


2.73 


0.93 


-0.14 


-0.92 


0.65 


0.76 


0.06 


0.78 


0.67 


0.10 


1.34 


19. You are under pressure from other people 


2.10 


1.03 


0.45 


-1.02 


0.58 


0.63 


0.38 


0.36 


0.47 


0.52 


0.50 


20. You feel discouraged * 


1.92 


0.84 


0.76 


0.11 


0.53 


0.63 


0.82 


<0.01 


0.68 


1.45 


<0.01 


21. You enjoy yourself (r) * 


2.12 


0.95 


0.38 


-0.85 


0.39 


0.52 


0.72 


-0.03 


0.49 


1.00 


-0.04 


22. You are afraid for the future 


2.40 


1.02 


0.14 


-1.07 


0.35 


0.38 


0.38 


0.10 


0.21 


0.43 


0.11 


23. You feel you're doing things because you have to (...) * 


1.98 


0.89 


0.63 


-0.34 


0.51 


0.50 


0.49 


0.13 


0.35 


0.66 


0.16 


24. You feel criticized or judged * 


1.82 


0.86 


0.78 


-0.22 


0.48 


0.50 


0.62 


0.03 


0.41 


0.81 


0.04 


25. You are light hearted (r) * 


1.97 


0.80 


0.44 


-0.38 


0.42 


0.53 


0.94 


-0.21 


0.65 


1.59 


-0.35 


26. You feel mentally exhausted * 


2.52 


0.96 


0.10 


-0.94 


0.64 


0.75 


0.25 


0.61 


0.64 


0.42 


1.01 


27. You have trouble relaxing * 


2.27 


1.03 


0.37 


-0.97 


0.60 


0.70 


0.35 


0.47 


0.58 


0.55 


0.72 


28. You feel loaded down with responsibility * 


2.53 


0.94 


0.09 


-0.90 


0.67 


0.77 


0.17 


0.69 


0.67 


0.29 


1.21 


29. You have enough time for yourself (r) * 


2.97 


0.89 


-0.52 


-0.52 


0.47 


0.61 


0.15 


0.53 


0.42 


0.20 


0.70 


30. You feel under pressure from deadlines * 


2.80 


0.95 


-0.19 


-1.01 


0.58 


0.67 


0.03 


0.70 


0.52 


0.04 


1.01 



r Kt-i - item-rest. G = second-order factor. & F 2 = first-order factors. h 2 = communality. & a 2 = IRT discrimination. r = reversed, "conserved for later analyses. 
doi:1 0.1 371 /journal.pone.0087071 .t002 



congeneric model, the estimates of reliability obtained for G were 
0.95; with 0.91 for F! and 0.93 for F 2 . The mean inter-item 
polychoric correlation for the twenty-four selected PSQ items was 
0.42. Item-rest values were positive and high, with an average of 
0.59. All the items were highly and positively correlated to the 
belonging factor calculated by EAP, with an average of 0.65 over 
F[ and 0.70 over F 2 . 

Convergence 

Table 4 shows the convergence values for the PSQ, hierarchical 
bi-factor structure scores calculated by EAP. F! and F 2 presented a 
correlation of r= 0.62 and both had high associations with regard 
to anxiety, depression and exhaustion. However, Fi presented 
higher values with cynicism, positively, and with resilience and 
efficacy, negatively. 

Discussion 

Despite the fact that perceived stress has been evaluated among 
dental students [9] , as far as we are aware, this is the first factorial 



study of the PSQ among dental students. It is also the first study to 
examine its internal consistency model as well as possible 
interrelatedness with burnout, anxiety, depression and resilience. 
Other studies have evaluated its structure in other samples [27,28], 
but the methods used did not respect the true ordinal nature of the 
variables, as we have. Our results provide evidence of a clear two- 
factor structure of the PSQ in dental students ('frustration' and 
'tenseness'), while it was also possible to use a single general factor 
('perceived stress'). Overall, the questionnaire had good psycho- 
metric properties, with adequate reliability and good convergence 
values, although it was advisable to discard some non-discrimina- 
tive items. We found that the congeneric model was the optimal 
model to measure its internal consistency. Interestingly, the results 
revealed different relationship patterns between the perceived 
stress factors and the other constructs. 

The main strength of the present study is that generalizability 
was enhanced because it was conducted using an high-stress-risk 
sample [4,7-9], from two different universities in two Spanish 
regions, and these groups exhibited similar response rate. It is 
interesting to highlight the fact that the study was carried out 
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.96 




.82 




.56 


5. Feel lonely 


.32 






.42 


7. Do not like things 


.18 






.72 


9. Not attain goals 


.52 


.79 


10. Feel not calm 


.63 


.74 


12. Feel frustrated 


.55 


.74 




13. Not energy 


.55 


.63 




17. Unprotected 


.40 






.81 


20. Discouraged 


.65 






.70 


21. Not enjoying 


.48 






.58 


23. Have to do things 


.34 






.60 


24. Feel judged 


.36 






.73 


25. Not light hearted 


.53 






.57 






2. Too many demands 


.33 






.63 


4. Many things to do 


.40 






.74 


8. Feel tired 


.54 


.38 


11. Too many decisions 


.14 


.82 




14. Feel tense 


.68 


.71 




16. Be in a hurry 


.50 


.80 




18. Many worries 


.64 






.84 


26. Mentally exhausted 


.70 






.80 


27. Not relaxing 


.65 






.85 


28. Responsibilities 


.73 






.67 


29. Not enough time 


.45 






.70 


30. Deadlines 


.49 







Figure 1. Construct validity of the PSQ hierarchical bi-factor structure. The circles represent latent constructs and the rectangles are 
observable variables. The factor weightings are over the one-way arrows and the percentage of explained variance for each observable variable over 
the boxes (standardized estimates). 
doi:10.1371/journal.pone.0087071.g001 



during the period of final exams, a well-known source of distress, 
which may make the results more relevant [66] . The response rate 
was high and the participants did not differ significandy from non- 
participants with regard to age, gender or years of study. 
Moreover, an independent researcher supervised the data 
transcription process to control for errors, and the analysis method 
respected the true nature of the variables used. The main 
limitation of the study was the use of a cross-sectional design 



because it did not permit the analysis of causal hypotheses. 
Another limitation could proceed from the instruments used, 
because they are not the only questionnaires used to measure such 
constructs. The use of other questionnaires might have produced 
slightly different results. 

The participants in the study were young adults. Most of the 
individuals were women who did not have children. The majority 
of students also did not receive financial assistance and were not 
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Table 3. Internal consistency models for the PSQ hierarchical 
bi-factor structure. 







Factors/Models 


R 


GFI 


AGFI 


RSMR 


NFI 


RFI 


Perceived Stress 


Congeneric 


0.95 


0.97 


0.97 


0.07 


0.97 


0.96 


Tau-equivalent 


0.95 


0.90 


0.89 


0.15 


0.88 


0.88 


Parallel 


0.95 


0.89 


0.90 


0.13 


0.87 


0.88 


Frustration 


Congeneric 


0.91 


0.99 


0.98 


0.05 


0.98 


0.98 


Tau-equivalent 


0.91 


0.95 


0.94 


0.11 


0.93 


0.93 


Parallel 


0.91 


0.95 


0.94 


0.10 


0.92 


0.93 


Tenseness 


Congeneric 


0.93 


0.99 


0.99 


0.05 


0.99 


0.99 


Tau-equivalent 


0.93 


0.95 


0.94 


0.12 


0.94 


0.94 


Parallel 


0.93 


0.95 


0.94 


0.10 


0.93 


0.94 



R= Reliability; GFI = Goodness of Fit Index; RSMR = Root Mean Square of the 
Standardized Residuals; AGFI = Adjusted Goodness of Fit Index; NFI = Normed 
Fit Index; RFI = Relative Fit Index. 
doi:1 0.1 371 /journal.pone.0087071 .t003 



employed. On average, the responses of the participants were not 
extreme, although the values for tenseness and anxiety were 
moderately high. Comparatively, the scores for frustration, 
burnout and depression were slightly lower. This findings can be 
understood by considering the pressure that the students 
experienced owing to the proximity of final exams. 

One of the most salient findings of this study is the clear 
hierarchical bi-factor structure shown by the PSQ among Spanish 
dental students. In this population, high correlations were 
observed between one second-order factor and two first-order 
factors. The second order factor (G) refers to 'perceived stress' as a 
general factor, as used in the original proposal [25,26] and in other 
studies [27,28]. The first of the first-order factors (Fi), which we 
referred to as 'frustration', mainly included items from the original 
'lack of joy' and 'worries'. The second of the first-order factors (F 2 ), 
named 'tenseness', mainly included items from the original 



'tension' and 'overload' [25]. In other words, F| consisted of the 
stress perceived as lack of joy and worries, and F 2 consisted of the 
external stressor of demands and the stress reaction of tension [28]. 

In general terms, the behaviour of the items was adequate, with 
high and positive item-rest values, although their distributions 
suggest a non-linear analysis, as was expected. All the items 
weighted strongly and positively in G factor. However, the IRT 
discrimination values advised us to reject some items because they 
were unable to differentiate adequately between F[ and F 2 . For 
these reasons, the 24 selected items were strong and positively 
weighted in the belonging factor and the model fit was very good. 
These items correlated highly with one another and internal 
consistency values were high, for G as well as for F[ and F 2 . Each 
item seemed to be measuring the corresponding latent variables, 
with possibly different degrees of precision and different amounts 
of error. 

We found that G was significantly related to all the considered 
constructs, and therefore this finding supports the hypothesis that 
burnout syndrome could mediate the link between perceived stress 
and the occurrence of emotional disorders [67,68], although it is 
well known that there are many others sources of stress, different 
from those related to work or workers in training. In other words, 
occupational stress could be an important risk factor for the 
development of anxious and depressive symptoms in dental 
students through burnout syndrome [69-71]. Thus, dealing with 
stress problems at an early stage of training may help to promote 
resilience and reduce burnout and mental health problems in the 
longer term [72]. Nevertheless, in order to reach a full 
understanding of the process, it will be necessary to continue with 
research until the hypothetical relationships put forward can be 
fully brought to light. It will be necessary to make use of designs 
that allow possible causal relationships to be explained, particu- 
larly if we are interested in developing effective lines of 
intervention, given the lack of these in a setting that is so in need 
of them. 

In addition, the pattern of relationships observed between the 
PSQ, first-order factors and the other constructs suggests that while 
both F[ and F 2 seems to be strongly associated with exhaustion, 
anxiety and depression, F! could be more positively related than 
F 2 to cynicism, and more negatively related to efficacy and 
resilience. In general, resilience increases the feelings of satisfaction 



Table 4. Convergence values for the PSQ hierarchical bi-factor structure. 





rg 


Md 


SD 


1 


2 


3 


4 


5 


6 


7 


8 


1. Perceived Stress 


0-1 


0.45 


0.19 


















2. Frustration 


0-1 


0.34 


0.20 


0.88* 
















3. Tenseness 


0-1 


0.56 


0.22 


0.86* 


0.62* 














4. Anxiety 


0-9 


5.30 


2.65 


0.60* 


0.50* 


0.56* 












5. Depression 


0-9 


3.39 


2.31 


0.67* 


0.64* 


0.56* 


0.63* 










6. Resilience 


0-40 


27.81 


6.74 


-0.48* 


-0.60* 


-0.28* 


-0.21* 


-0.42* 








7. Exhaustion 


0-30 


13.49 


7.49 


0.76* 


0.64* 


0.70* 


0.53* 


0.66* 


-0.34* 






8. Cynicism 


0-24 


5.57 


4.74 


0.43* 


0.48* 


0.26* 


0.1 3t 


0.40* 


-0.32* 


0.45* 




9. Efficacy 


0-36 


24.85 


5.62 


-0.28* 


-0.43* 


-0.08 


-0.05 


-0.21* 


0.54* 


-0.16* 


-0.38* 



rg = range. Md = mean. SD = standard deviation. Perceived Stress (G), Frustration (F,) and Tenseness (F 2 ) from PSQ. Anxiety and Depression from GADS; Resilience from 

10-item CD-RISC; Exhaustion, Cynicism and Efficacy from MBI-SS. PSQ descriptives were calculated as: (raw score-24)/72. Frustration and Tenseness descriptives were 

calculated as: (raw score-12)/36. Convergence values are Spearman's R correlations (PSQ factors calculated according to the Bayes 'Expected A Posteriori'). 

*p<0.001. 

*p<0.01. 

f p<0.05. 

doi:1 0.1 371 /journal.pone.0087071 .t004 
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and commitment that promote the empowerment of and 
beneficial outcomes for students and even workers [73-77]. In 
accordance with the demands-resources model [78], people who 
suffer from burnout experience a progressive decline in commit- 
ment to their studies over time [79]. This is perhaps why resilience 
constitutes a coping reservoir that influences the long term 
functioning of university students [17,72). However, our results 
suggest that interventions focused on promoting resilience may be 
of benefit in overcoming frustration more than tenseness, while 
tenseness seems to be more important in this population. 
Combining resilience interventions with relaxation interventions 
could possibly be the best choice for dental students. 

Conclusions 

The PSQ_ showed a hierarchical bi-factor structure among 
Spanish dental students. Using the questionnaire as a unidimen- 
sional scale may be useful for discerning perceived stress levels, 
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